Random Seeking: A General, Efficient, and Informed Randomized Scheme for Dynamic Load Balancing

We propose a completely general, informed randomized dynamic load balancing method called random seeking (RS) suitable for parallel algorithms with characteristics found in many search algorithms used in artificial intelligence and operations research and many divide-and-conquer algorithms. In it, source processors randomly seek out sink processors for load balancing by flinging “probe” messages. These probes not only locate sinks, but also collect load distribution information which is used to efficiently regulate load balancing activities. We empirically compare RS with a well-known randomized dynamic load balancing method, the random communication (RC) strategy, by using them in parallel best-first branch-and-bound algorithms on up to 512 processors of an nCUBE2 multicomputer. We find that the RC execution times are more than those of RS by 8– 67% when used to perform combined dynamic quantitative and qualitative load balancing, and by 5–74% when used to perform just dynamic quantitative load balancing.